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We propose an extension of the zone-based algorithmics for analyzing timed automata to handle 
systems where timing uncertainty is considered as probabilistic rather than set-theoretic. We study 
duration probabilistic automata (DPA), expressing multiple parallel processes admitting memoryfull 
continuously-distributed durations. For this model we develop an extension of the zone-based for- 
ward reachability algorithm whose successor operator is a density transformer, thus providing a so- 
lution to verification and performance evaluation problems concerning acyclic DPA (or the bounded- 
horizon behavior of cyclic DPA). 



1 Introduction 



Timed automata [4] handle temporal uncertainty in a set-theoretic manner consistent with the worst- 
case spirit of safety-critical verification. Performance evaluation of systems of a less dramatic nature 
is typically based on a stochastic interpretation of temporal uncertainty. A well-studied class of such 
systems are continuous -time Markov chains (CTMC) where durations are distributed exponentially and 
model-checking against temporal properties is well understood [7]. More general distributions fall under 
the category of generalized semi-Markov processes (GSMP) |[T8l [T71 [l31l and other similar models such 
as stochastic timed automata |[T6l[T0ll or stochastic Petri nets ET1 I51. Good overviews of these issues can 
be found in ifT^fTTl . Some approaches for verifying such systems against qualitative [3] and quantitative 
ll22l properties have been proposed based on partitioning the state space into equivalence classes in the 
spirit of the region graph and performing the analysis on the finite quotient which can be viewed as 
a discrete-time Markov chain. Although the region graph underlies the fundamental decidability results 
for timed automata, it is not used in any existing verification tool, due to its prohibitive size. Verification 
tools Il27ll24l use reachability computation on zones lfl9l . a class of polyhedra that represent reachable 
sets of states and clock valuations^ 

We extend the zone-based reachability computation to handle timed automata with probabilistic du- 
rations. We use a variant of stochastic timed automata that we call duration probabilistic automata 
inspired by the class of timed automata encountered while modeling scheduling problems [1]. Such au- 
tomata can model tasks admitting precedences and resource constraints, with the duration of each task 
being probabilistically distributed. We focus on uniform distributions but the proposed approach will 
work with any polynomial distributions with bounded support. To analyze such systems we decorate 
zones with clock densities, and define successor operators that act as density transformers that allow 
us to compute the clock distribution upon taking a particular transition from state q based on the clock 
distribution at the entrance into q. As a result we can assign probabilities to interesting subsets of the 
timed language generated by the automaton. 

'Theoretically the number of zones can be even higher than the number of regions but in practice it is much lower. 
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Figure 1: A Minkowski sum of intervals versus a convolution of two probability density functions. The 
area outside the dotted vertical lines corresponds to low-probability behaviors that can be ignored in 
certain circumstances. 

The rest of the paper is organized as follows. Section 2 is a self-contained introduction to the mod- 
eling of timing uncertainty in concurrent systems and its algorithmic analysis. In Section 3 we define 
duration-probabilistic automata. Section 4 is devoted to a summary of the reachability graph construction 
used to compute the semantics of timed automata. In Section 5 we present our contribution, the extension 
of this technique for DPA using density transformers while Section 6 mentions related and future work. 



2 Timing Uncertainty: Modeling and Analysis 

Discrete concurrent processes can be analyzed at different levels of abstraction with respect to time. To 
illustrate this point consider two concurrent systems, one that performs two tasks sequentially and one 
that preforms a third task in parallel and let events a, b, and c denote the respective terminations of these 
tasks. At the most abstract level one assumes nothing about the relative durations of the processes and 
hence all the sequences in the shuffle ab\\c = {abc,acb, cab} are considered feasible. The first refinement 
of the model is provided by models such as timed automata or timed Petri nets, where the durations of a, 
b and c are specified to be bounded in the intervals [Z a ,M a ], [h,Ub] an d [l c , u c ], respectively. In this model, 
knowing, for example, that l c > u a we conclude that c cannot occur before a and hence cab is impossible. 
Likewise, abc is impossible when u c < l a + h- 

While this refinement of the untimed model adds a lot of information, this set-theoretic nondeter- 
minism which states only what is possible but does not quantify the likelihood of different possibilities, 
is still too qualitative for certain purposes as the following example demonstrates. Consider a sequence 
of k processing steps, each of which with duration in [/,«]. From a purely "measureless" set-theoretic 
viewpoint, the termination time of the whole sequence of steps can be anywhere in [kl,ku]. Intuition tells 
us, however, that a duration of ku, whose realization requires that each of the steps takes the maximal 
time to terminate, is less likely than, say, an "average" duration of k(l + u)/2^ On the other hand if 
we interpret the interval [l,u] as, say, a uniform distribution with density l/(u — I), the total duration of 
the fc-step sequence is still restricted to the interval [kl,ku], but with probability which is larger in the 
middle of the interval and smaller toward the boundaries. In a nutshell, this is the difference between 
a Minkowski sum of intervals [Z,w] © [l,u] and the convolution * ¥2 of two functions defined over 
those intervals, see Fig. [T] Assigning probabilities to the runs of the automaton we can, for example, 
distinguish between different degrees of property violations or compute the expected value over all runs 
of some performance measure. 

The use of automata with clocks has some advantages over the standard language of stochastic pro- 
cesses, in particular, the ability to express more sophisticated synchronization mechanisms between pro- 
cesses, such as schedulers that resolve resource conflicts. These are expressed naturally in the language 

2 Another example of a more discrete nature is the modeling of computer memory access where worst case duration (cache 
miss) is orders of magnitude larger than the normal case (cache hit) and if we want to be conservative and assume that both 
cases are possible in each and every instance, our performance estimation will be overly pessimistic and practically useless. 
Timed automata with probabilities on transitions have been studied in I20II23I . 
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Figure 2: A process that takes time: (a) standard description; (b) decoupling the non-deterministic choice 
from the end transition. 



of states and transitions while translating them into conditionals based on inequalities over values of 
random variables may be cumbersome. Computationally, a state -based approach provides for iterative 
forward or backward computations, for both analysis and scheduler synthesis, more flexible than meth- 
ods based on a holistic analytical solution^] For timed automata, this iterative computation works on sets 
of clock valuations (zones) |[T9ll27ll24l that each qualitative sequence of events may lead to, where clock 
values eliminate qualitative behaviors which are infeasible due to timing. In the probabilistic setting, we 
decorate zones with additional probability information concerning runs and clock values. Thus we can 
eliminate classes of behaviors which are feasible but unlikely. Hopefully, the non-negligible overhead 
associated with computing probabilities will be compensated by the liberty not to explore paths of low 
probability. 

We consider processes constructed from very simple components such as the automaton of Fig.^a). 
Such a process is in a waiting state, until it takes a start transition and moves to an active state. Clock x, 
which is set to zero upon the transition, measures the time elapsed since the activation. A start transition 
is instantaneous and is initiated by some external scheduler/supervisor. The timing of an end transition 
is based on the clock value and the temporal guard </> (x), which in the case of timed automata, is simply 
the condition x € [l,u]. In duration probabilistic automata we associate a probability density with the 
duration of each step which is technically expressed as the distribution over the values of clock x when 
the end transition is taken (note that once started, a process cannot be aborted). We want to analyze the 
behavior of multiple such systems running concurrently, each with its own clock. 

We use a slightly modified (but equivalent) version of the basic automaton, as shown at Fig. |2jb). 
Rather than having the start transition deterministic and delegating the non-determinism to the end tran- 
sition, we use an auxiliary variable y which is assigned non-deterministically upon start and which should 
be equal to x upon end. In the set-theoretic setting this means an assignment y G [/,«] while for DPA this 
means drawing a value for y according to 0, which we denote by y := 0(). 

The fundamental phenomenon in the analysis of continuous-time stochastic processes is that of a race 
which occurs in a global state where two or more processes are active. We would like to know which 
process terminates first, in other words, via which of the pending end transitions will the automaton 
leave the state. The outcome of a race depends on two factors: the random choices of the respective task 
durations (the y variables) and the values of the clocks upon entering the global state. Fig. [3] shows a 
fragment of a global automaton representing two parallel processes, both active at state q. Clock xi was 
reset upon entering q, while clock x\, corresponding to a different process that has not yet terminated, was 
reset in a preceding global state. The gap between the two starting times is maintained by the difference 
x\ — X2 which remains constant throughout the sojourn in q. The larger is this difference, the more likely 
is clock x\ to satisfy its temporal guard by reaching y\ before xi reaches j20 



3 We use automata here as a generic term for discrete transitions systems. Some of the advantage attributed to them in terms 
of modeling expressivity and analysis techniques apply, at least in principle, to other similar formalisms such as Petri nets for 
which an approach similar to ours has been developed in 1261 . see Section 6] 

4 It is interesting to note that in the stochastic processes literature 1 15 1 the role of x and y is taken by a single timer z = y — x 
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Figure 3: A race. 



In the probabilistic setting, this is rephrased as follows. Suppose we enter a global state with some 
probability over clock values, and in this state there are several pending end transitions guarded by 
probabilistically-chosen durations. The probabilities over the clock values upon entrance together with 
the probabilities over the durations determine the probability that a certain transition wins the race and is 
taken, as well as the probabilities over the clock values upon taking each of the transitions. We develop a 
computational scheme for computing for every finite sequence of events the probability that it occurs and 
the probability over the clock values upon the occurrence of its last event. Technically this is achieved via 
the concept of a density transformer which extends the symbolic successor operator of timed automata 
(which deserves to be called a subset/zone transformer) to an operator on (partial) densities over clock 
values. 

3 Definitions 

Throughout this paper we use T = [0,°°) as a time domain on which we define probabilities. We use a 
fixed set of clock variables X = {x\ , . . . ,x n } all ranging over T or a bounded subset of it. 

Definition 1 (Clock Constraints and Zones) The set of clock constraints over X, is defined by the fol- 
lowing grammar: (p ::= true \%i -<k\xi—xj~<k\(pA<p', where Xi,Xj Gl, k £ N and -<£ {<,<,=,>,>}. 
The set of points satisfying a clock constraint is called a zone 

Each zone is a convex polytope in some dimension m<n defined as the intersection of half-spaces which 
are either orthogonal [xi -< k) or diagonal {xj —Xj -< k) with integer k. There are finitely many zones in any 
bounded subset of T" or any of its subspaces. We use _L to denote the zone associated with dimension 
zero (where no clock is active). 

Definition 2 (Time Densities) A piecewise-continuous function : T — >• T is a time density if it satisfies 



A density has a bounded support [a,b] C T if<\){x) / 0f>TS [a,bj. A bounded support density is uniform 
if(j)(x) = l/(b-a) when x G [a,b]. 

The generalization to higher dimension is: 



which is a clock going with derivative — 1 to zero, after being assigned a random duration. The difference between the two 
formulations is that ours distinguishes the information that is observable at any time, the value of x, from the information that is 
observed only upon termination, the actual duration y. This two- variable representation may provide for more refined dynamic 
schedulers that can base their decisions on the value of x, as demonstrated in (T|. 
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Definition 3 (Clock Densities) A function y : T m — > T is a clock density if it satisfies 

poo poo 

/ ... / y/-(Ti,...,T m )<iTi ...dx m = 1. 

JO JO 

We will consider clock densities whose supports are zones^ 

Abusing terminology we call y a partial density if the above integral is smaller than 1 . 

To define the behaviors of our automata we will use timed words (the time-event sequences of flU) 
over an alphabet E of events which will correspond to the various start and end actions. 

Definition 4 (Timed Words and Languages) A timed word over a finite alphabet E is a concatenation 
of the form t, = t\ ■ w\ ■ H ■ W2 • ■ ■ where tj G T and w; G E + . The untiming oft, is jU ) = W\ ■ w% ■ ■ ■ and 
its duration is X(£,) = A timed language is a set of timed words. 

Intuitively this object represents an alternation between passages of time of duration t{, followed 
by sequences w, of one or more instantaneous events. The events will be start and end transitions and 
time passages correspond to time elapsing in active states. All events in W{ occur at the same absolute 
time instant Ly=i tj Dut in order not to extend the alphabet to 2 Z we will consider them as occurring 
sequentially. We use e for the empty word. A timed word such that ju(^') = ju(^) agrees with £ on 
the order of events. All such behaviors form an equivalence class [£] that we sometime refer to as a 
qualitative behavior. 

Duration probabilistic automata (DPA) constitute a well-structured class of timed automata obtained 
as products of simple DPA and a scheduler. They can model most situations encountered in the analysis 
of scheduling problems such as job-shop or task-graph [ Q and are free from notorious anomalies such as 
Zeno behaviors. For economy of expression, we use as our building blocks processes that admit several 
processing steps where the end transition of step j leads to the waiting state of step j + Although 
practically, the same clock can be reused in subsequent steps, conceptually we prefer sometimes to view 
each step j as using a distinct clock* 7 . Let N = {1, . .. ,n} and K = {1, .. . ,k}. 

Definition 5 (SDPA) A simple duration probabilistic automaton ( SDPA ) of k steps is a tuple si = 
(E, Q,X,Y,A,q l ) where E = E^ l±)E e is the alphabet of start and end actions with E s = {s l , . . . ,s k } and 
E e = {e l , . . . ,e k }. The state space is an ordered set Q = {q l ,q l ,q 2 , . . . ,q k ,q k+l } with qi states consid- 
ered idle and states are active, X = {x l ,x k } is a set of clock variables and Y = {y [ ,y k } is a 
set of auxiliary random variables, each distributed according to a bounded and uniform time density (j) J . 
The transition relation A consists of two types of transitions: 

1. Start transitions: for every idle state q J , j G K, there is one transition of the form (q J ,s J ,{x J },q J ). 
When the transition is taken, clock x ] is reset to zero and becomes active. Such transitions take no 
time; 

2. End transitions: for every active state q 3 , j G K, there is a transition of the form {q } ,x J = 
y J ,e J ,q J+1 ). This transition renders clock x inactive. 

State q l is the initial state of 

The SDPA just defined is acyclic. A cyclic version of this definition, employs addition modulo k with 
the last transition going back to q l , see Fig. [4j In this paper we restrict ourselves to acyclic automata. 

The operational interpretation is the following: for each step j we draw a duration y according to 
<pj. Inside an active state ql, clock x-i advances with derivative 1 and the end transition is taken when 



5 More precisely, due to resets that put all the probabilistic mass of some clocks at zero, we have to deal with hybrid objects 
that combine discrete and continuous probabilities and can be framed in terms of densities using impulse functions. 



38 



Duration Probabilistic Automata 



jf 1 :=*'() 




s 

x ] :=0 


!/:=^() 






I 





Figure 4: A simple DPA: acyclic and cyclic (dashed transition). 

x-i = y , that is, y time after the corresponding start transition. A generalized state (configuration) of 
the automaton in an active state is a pair (q,v) consisting of a discrete state and a clock value v which 
represents the time elapsed since the last start transition. Note the difference between transition labels si 
and e 7 : the former is an external command coming from a scheduler outside the SDPA, while the latter 
is emitted by the SDPA itself when it terminates a step within a randomly chosen duration. When such 
a scheduler is not specified, the automaton can be viewed as non-deterministic, generating behaviors of 
the form 

r 1 - S l -t 1 -e x -r 2 -s 1 -t 2 -e 2 -- S k -t k -e k -~> 

with each r J G T being an arbitrary waiting period and each t-* is in the support of 

Duration probabilistic automata (DPA) are obtained by composing a set = (£,-, Qj,Xj, A[,q} 
of SDPA with a scheduler. To simplify notations we assume all ^ to admit the same number k of 
steps. The event alphabet is the union of the event alphabets E;, that we write as £ = L s l±l L e with 
T, s = {sf : i € N,j € K} and L e = {ej : i € N,j € K}. The state space of the product automaton is 
Q = Q\ x ••• x Qn- The composition of automata, which is fairly standard in the non-deterministic 
setting, often employs an interleaving semantics where independent transitions can occur in any order. 
Applying this approach to several start transitions that take place simultaneously, introduces an annoying 
artificial non-determinism that we avoid by combining all transitions that occur simultaneously into 
a single transition. However in order to maintain the semantics of the automaton as a set of timed 
words over Z we will associate with such a transition a unique sequence of labels. This is done via a 
sequentialization function which maps every £C£ into a sequence a(E) 6 £ + consisting of the elements 
of E concatenated according to some fixed order relation over the alphabet. We say that transition s\ is 
enabled in global state q if the i th component of qisq J r 

Definition 6 (Scheduler) A scheduler for a set {M}ieN of SDPA is a function S : Q — > 2 Zs , satisfying: 

• sj £ S(q) only ifsj is enabled in q; 

• S(q) = only if q is the global final state or admits at least one active component. 

The scheduler plays two roles in our model. First, it guarantees mathematical sanity with a single 
run for every value of the random variables and a non-blocking behavior where all prefixes of runs have 
continuations that reach the final state in a bounded amount of time. In a world of unlimited resources 
where each SDPA may progress independently, S(q) is the set of all transitions enabled in q and the 
scheduler is restricted to this mathematical role. The more interesting case is when the scheduler has 
to resolve resource conflicts and keep some processes waiting while giving priority to others. Abusing 
notation we say that i € S(q) if s\ G S(q) for some j. 

Definition 7 (Duration Probabilistic Automata) A duration probabilistic automaton (DPA) is a com- 
position stf = srf\ o ■ ■ • o stf n o S = (Q,X, YA, q°) of n SDPA and a scheduler. The state space is Q C 
Q\ x '"Qn with initial state q l = (q\, . . . ,q\) the set of clock^is X = U;X; and the auxiliary variables 

Since at any time there is at most one clock active for each stf,, we will sometimes refer to the set of clocks as {x\ , . . . ,x n } 
where x,- refers to some x\ depending on the state of .iz^. Likewise we will compare it with y; denoting the appropriate y/. 
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Y = \Ji Yj. The transition relation A consists of two types: multiple start transitions of the form (q, W,R, q') 
where w G is a sequence of labels and R is a set of initialized clocks, as well as end transitions of the 
form (q,Xj = >•,-,«,•, ^ / ), one for each a/j active in q. 

• For every state q = (qi,...,q n ) such that S(q) = E ^0we define a transition 

((q h ...,q n ),w,R,(p h ...,p n )) 

where w = Ot(E) is the sequentialization ofE andR = {xj : sj G E}. When i G" E pi = qt otherwise 
Pi = q'j where (qi,S(, G A,- is the corresponding start transition; 

• For every q such that S(q) = and for every i such that qi is active and (qi,Xj = yi,ej,q'j) G A, is 
its corresponding end transition, we define a transition 

((qu---,qi,---,q n ),Xi = yi,e i ,(qi,...,q' i ,...,q n )) 

This definition gives priority to the immediate start transitions while the pending end transitions are 
allowed only in a state where no immediate transitions are admitted by the scheduler. 

4 Behaviors and their Computation 

The set of all complete behaviors that a DPA srf may generate constitutes a timed language L = L(g/). 
The probabilistic semantics of srf is a probability distribution over subsets of L. We will not give at this 
point a detailed formal definition of this semantics but rather convey sufficient intuition to relate it to 
the zone-based computation that we develop in the sequel. For the sake of simplicity, we temporarily 
assume a most liberal scheduler which executes every sj immediately after ej 1 . The untiming jJ.(L) 
of the language consists of words satisfying some well-formedness condition, that is, jU (L) CM where 
M = Mi 1 1 • • • | \M n is the shuffle of the SPDA local languages, each of the from M; = {s] ■ e) ■ ■ ■ s\ ■ e\}. 
By construction, there is a one-to-one correspondence between sequences of events in M and complete 
paths in M '. Hence L can be written as a union UweM^w of languages, each corresponding to a subset of 
L corresponding to a particular order of events. Elements of L w are obtained from w by inserting time 
durations between the events. 

Each choice y of values for the duration random variables determines a unique behavior of the system 
that we denote <^ (y) and the probability of a set of behaviors is the probability of the y values that induce 
them. The density of this distribution at a complete timed word t, = t\C\, . . . ,t n k<3nk under a liberal 
scheduler is defined as follows. For every step (i,j) £ N x K, let rj be the sum of all duration occurring 
between sj and ej. Then the density at E, is: 

n <Vf/)- (i) 

ieNjeK 

Unfortunately ([TJ cannot be exported as is to the case of non-trivial schedulers where we have to resort 
to incremental computations that derive the probability of E, ■ t ■ a from the probability of its prefix § . To 
this end we need to consider incomplete behaviors that correspond to a word w in which not every s has 
been followed by a matching e. The probability of <^ • t ■ e for each of the pending end events depends 
on the probability of the corresponding step to terminate within a duration equal to the sum of t and the 
duration in <^ occurring after s and the probability of the other steps already started in w to terminate 
after that. 
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Figure 5: (a) Two commuting paths; (b) splitting a state into two copies according to the history. 



An incomplete behavior £ can be associated with two other objects, the first being the subset of 
L consisting of complete words having ^ as a prefix and the second is a global configuration of the 
automaton reached while generating A global state in a timed automaton is a mixture of active 
and idle local states with active clocks defined naturally according to the state, and this determines the 
dimensionality of clock space in that state. Thus a configuration is a pair (q,v) with v G T m for some 
m < n, and the time evolution inside the state consists of all active clocks advancing in the same pace, 
keeping the difference between any pair of active clocks constant throughout the sojourn in a state. The 
set of time predecessors of a clock valuation is n(v) = {v — t1 : t > 0}nT m , where 1 is a vector (1, . . . , 1) 
of dimension m. A configuration (q,v) can be reached via time passage only from configurations of the 
form (q,V) with V G 7t(v). 

Let us just comment on the issue of commuting paths in the automaton. Why can we merge two 
such paths into a single state despite their differing past histories? The reason is that the past events that 
occurred in different orders along the two paths are of two types: 1) events related to completed steps 
that do not affect the future beyond what is already encoded in the state; 2) start transitions of steps 
which are still active in q. These events do affect the future but the order of their occurrence is captured 
already, at a finer level of detail, by the values of the active clocks and their pairwise differences. This 
is illustrated in the two commuting paths depicted in Fig. |5ja), assuming step 3 to follow step 2 in the 
same SDPA. The qualitative languages associated with the paths are the singletons w\ = *i*2£i£2*3 and 
m>2 = 51*2^2*3^1 > respectively, while the qualitative language of the whole state q is *i*2(^i I ^2*3) and 
the only information that still affects the future is the time elapsed since *3, captured by a clock (see also 
|[25l ). Despite this fact, for convenience reasons, we split states according to their respective histories, 
that is, work with extended discrete states of the form (q,h) where h G £*. A transition from q to q' 
labeled by some w£l + thus extends into a transition from (q,h) to (q',h- w), and the transition graph 
of the automaton becomes a tree, see Fig.[5jb). 

We will present our method to compute the probabilistic semantics gradually starting with its support, 
which is the set of all timed words which are possible if we interpret each <p as an interval, as in timed 
automata. Although what is described in the sequel is standard material underlying the practice of TA 
verification tools Il27ll24ll . it is our perception that it is not sufficiently known to the more general public. 
We assume that for every component i active in state q, the duration of its corresponding step is distributed 
with a uniform density <j>i of support [a;,&,-]. We use R(v) to denote the setting to zero of clocks in R and 
the continuation of the clocks in v that are not in R. Note that by the definition of SPDA all clocks in R 
are inactive in q before the transition. 

Definition 8 (Steps and Runs) A step of a DPA srf is one of the following. : 

• A start step: (q,h,v) — > (q 1 ,h- w,v'),for some (q,w,R,q') G A such that v' = R(v); 
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• A time step: (q,h,v) — (q,h,v + Tl); for some T > such that for every i active in q, v, + T<Z?,-; 

• An end step: (q,h,v) (g,/z • e,-, v') where v, G [ai,6f] and v' is obtained from v by deactivating 

Xi. 

A run of the automaton is a sequence of steps which starts at (q , £,J_) a«<i alternates between single 
time steps and one or more transition steps. 

The behavior associated with a run is the timed word obtained by concatenating the labels (transitions 

p 

and durations) of its steps. We use the notation (q,h,v) — > (q,h ,v) to denote a run from (q,h,v) to 

(q ,h , v ) generating the timed word g (note that /j' = /z- A(^)). We also use the notation (q,h,v) — > (• • • ) 
to denote an infinite run starting from (q,h,v). For acyclic DPA, all such runs terminate with an infinite 
time step inside the final state. 

Definition 9 (State Languages) With every extended configuration (q,h,v) we associate the following 
timed languages: 

• The set of behaviors associated with runs whose last event is a o-labeled transition to (q,h- d): 

L-+°(q,h,v) ={$-c : (q\e,±) ^ (q,h-a,v)} 

• The infinite behaviors generated by runs that start from (q,h,v): 

L^(q,h,v) = {^.(q,h,v)^ (••■)} 

• The set of all infinite behaviors of £/ with prefixes in L^°(q,h,v): 

L(q,h,v) = L^°(q,h,v) • (q,h,v). 

Observation llf%£ L^°(q,h,v) then v,- is equal, for every i active in q, to the time elapsed since the 
last Si event in £ . 

Definition 10 (Symbolic States) An (extended) symbolic state is a triple (q,h,Z) with q G Q, /i G E* and 

Z is a zone of dimensionality compatible with q. 

Intuitively, Z will be the set of all possible clock values that runs along the path to (q,h) may have. We 
will lift the definition of state languages to symbolic states by letting L(q,h,Z) = \J veZ L(q,h,v). We 
associate with time passage and with every transition a successor operator over symbolic states. 

Definition 11 (Successor Operator) Successor operators admit three types: 

• Time successors: post' (q,h,Z) = (q,h,Z r ) where 

Z' = {V : 3v G Z3T G T (q,h,v) (q,h,v)} 

• Start successors: post s (q,h,Z) = (q',h- w,R(Z)) for every start transition 
(q,w,R,q')eA; 

• End successors: post e (q,h,Z) = (q',h ■ e,Z') for every transition (q,x = y,e,q') G A where Z' is 
obtained from Z by eliminating the appropriate de-activated clock. 

The reachability graph, also known as the simulation graph, is what timed automata verification tools 
E71 l24l compute as a symbolic representation of the semantics of the automaton. 
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Definition 12 (Reachability Graph) The reachability graph associated with a DPA gf is a graph of 
symbolic states obtained by successive application of successor operators to (q 1 , £, _L). 

The fundamental property of the reachability graph is the following. 

Theorem 1 A symbolic state (q,h,Z) is part of the reachability graph iff for every v £ Z, the language 
(q,h,v) is not empty. 

In other words there is a timed word ^ generated by the automaton with p(§) = h such that for every 
active component s^u the duration of the suffix of £, starting with the last S{ event is v ; -. Note that since 
all runs of a DPA have a continuation, L^°(q,h,v) ^ implies that L°^(q,h,v) ^ and L(q,h,v) ^ 0. 
Moreover, L(q,h,Z) is exactly Lj,, the set of timed words in L whose untiming is h. 

In the sequel we will extend the reachability graph with probabilities and work with symbolic states 
of the form (q,h,Z, y) where y is a partial density function over the clock values in Z, which can be 
used to compute the probability of L(q,h,Z) or its subsets. To this end we need to extend the successor 
operators to become density transformers. 

5 Density Transformers 

The major issue in our computational approach is to determine, in a state where several processes are 
active, the probability of each of the pending end transitions to be taken and how the clock values are 
distributed when the transition is taken. As an informal illustration consider state q in the automaton of 
Fig. [3] admitting two competing active processes whose durations are distributed with densities 0i and <j>2, 
respectively. Assuming both 0i and </>2 are uniform with a bounded interval support, their joint density 
l j ^2 ) = 01 (jl)<fc(j2) is supported by arectangle of the form [ai,&i] x [02,^2]- The clock values with 
which the state can be entered are restricted to the rectangle [0,bi] x [O,^] and the two transitions can 
be taken in the rectangles x [0,62] and [0,bi] x \a%,b^, respectively, see Fig.|6ja). Note that the 

points of exit need not be inside the (joint) support of (p. 

What is the probability Pi(u\v) that transition i is taken at some point u = (ui, U2), i.e., M; = yi, given 
that the state has been entered at some v? First of all, this probability is non-zero only if v G 7u(w), that 
is, v is a time-predecessor of u. Secondly, for transition 1 to be taken, it should be the case that process 1 
chooses duration u\ while process 2 chooses some y2 > W2 (the vertical thick line in Fig.[6jb)). Transition 
2 will be taken at u when process 2 chooses a duration U2 and process 1 some yi > u\ (the horizontal 
thick line in the figure). Thus p\{u\v) is obtained by summing up the duration probabilities above u 
and p2(w|v) by summing up the probabilities to the right of u. Note that pi(u\v) = p{u\V) for any other 
v 1 G 7t(u) and that for points like u' outside the support of <j>\ we will have pi (u'\v) = and p2(u'\v) = 1. 
Assuming that the state has been entered with some density y over clock values, we can sum up Pt(u\v) 
over v G 7l(u) according to y and obtain the expected p,-(«) as well as new densities y reflecting the 
distribution of the clock values upon taking each of the transitions. 

With every extended state (q,h) in which m processes are active we associate a partial density func- 
tion of the form y{x\ ,x m ,y\ ,y m ) whose intended meaning is to capture the probability over clock 
values upon entering the state. Although the y variables are static and do not vary during execution, we 
need to keep them in the picture because they do not distribute evenly as time goes by. In other words, 
certain combinations of choices of durations will make some transitions impossible. We associate density 
transformers with every start and end transition as follows. 

Start: Let q be a state with / active components and let s be a start transition which activates processes 
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Figure 6: A race: (a) a state can be entered at any point in the shaded area; transition can be taken only 
in the darker area; (b) the probabilities pi and pi. 

{/ + 1, . . . ,m}Q We associate with s the density transformer <9J such that y' = if 
\j/(xi,...,xi,0,...,0,y h ...,yi,yi +h ...,y m ) = y(x\,...,xi,yi,...,yi) 

■${yi+\,---,y m ) 

with^(yi +1 ,...,y m ) = <t>i + i(yi +1 )---<t> m {y m ). When one of {x M ,...,x m } is non-zero, y/ = 0. This oper- 
ation just reflects the setting of the new clocks to zero and the introduction of their respective durations. 
End: For every end transition e; outgoing from a state q with m active processes we define two density 
transformers 3? n and . As explained previously, the transformer 3F r . computes the clock density at 
the time when process i wins the race, given the density was y upon entering the state. It is defined as 
Wi = if 

Wi(xi,...,x m ,yi,...,y m ) = 




Y(xi -z,...,x m -z,y u ... ,y m )dT if x t = y t A V/' / i x v < y t 

otherwise 



The transformer 3T±_., which just deactivates clocks, and projects it away from the clock space is defined 
as y' = =5j_, (v) if 

y(x\,. . . ,Xi-i,Xi+i,. . ■ ,x m ,yi,. . . ,yi-i,yi+i, . . . ,y m ) = 

/ y(xi, . . . ,Uj, . . . ,x m ,yi, . . . ,Uj, . . . ,y m )dui. 

We can now define a probabilistic version of the successor operators. Note that for timed automata 
we had a unique time successor operator for each state, while for DPA time successor operators are 
specific for each of the transitions that participates in the race. A probabilistic symbolic state is a tuple 
{q,h,Z,y). 

Definition 13 (Probabilistic Successor Operator) Probabilistic successor operators admit two types: 

7 The restriction to these indices is just to simplify notation. Recall also our previous remark that our probabilities are in 
reality hybrid, mixing discrete probabilities and distributions. 
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• Start successors: post s (q,h,Z,Y) = (q* ,h-w,R(Z),\j/) for every start transition (q,w,R,q r ) £ A 
where y/ = ^(y/); 

• End successors: post e '(q,h,y,Z) = (q' ,h- a, Z' ,\lf') for every transition (q,x = y,ei,q') £Awhere 

= ^(^rXv)) an d Z' is the support of\j/'. 

The probabilistic reachability graph is computed by starting with the initial probabilistic symbolic state 
(q , £,_L,y/jJ and then applying the appropriate successor operators. Computing this graph, as in the 
case of timed automata, allows us to compute everything of interest for DPA as we show below. 

Recall that every y valuation induces a complete run ^(y) with an untiming h. Grouping all the y 
values resulting in the same h we have a mapping from the duration space W k to the finite set ~L nk which 
defines the probability of each path. To extend this notion to incomplete behaviors one could define a 
sequence of functions {f a : a = 0, . . . ,nk} over the duration space, each mapping y into a prefix t, (y) a 
of ^(y) admitting exactly a discrete transitions. As mentioned earlier, to compute f a+ \ from f a it is 
sufficient to know the qualitative prefix h a and the time elapsed since the non-terminated start events. 
For each a we have then a hybrid (discrete-continuous) probability distribution on L a x W" which can 
be expressed as a finite set of densities {r\h '■ h £ Our main claim is that if (q,h,Z, y) is part of the 
probabilistic reachability graph then 



This holds trivially for the root (q l ,£,J-,Y±) which corresponds to rj e where all the probability is 
concentrated in the empty sequence. The inductive step, showing that if a node (q,h,Z,\j/) satisfies 
ilh(x) = J Wi x ,y)dy than any successor (q' ,h' ,Z' , y/) satisfies r\h'{ x ) = J W'(x,y)dy, is immediate for a 
start successor because it just concatenates some ^-labels without changing probabilities. For an end 
successor e,, observe that for every v 6 Z and V G Z', the corresponding run leads from (q,h,v) to 
(q',h ■ ei,v'), concatenating to the language a timed word z • e\ with z = v • — v,- and the probability of V, 
the time elapsed since the remaining uncompleted start events, is captured by y'. 

Thus we can compute the probability for each interesting set of paths, for example those in which 
some event precedes another. Moreover, by adding an auxiliary clock which is never reset and measures 
absolute time, we can retrieve the evolution of these probabilities over time and compute the distribution 
and expected value of the total termination times. This provides for an effective comparison between the 
performance of different scheduling policies. 

6 Past and Future Work 

We have shown how timed automata verification techniques can be extended to handle durations which 
are distributed probabilistically. We conclude by mentioning some related work as well as some of the 
many open issues that remain. 

The works closest to ours are those of Alur and Bernadsky ||2l |9l and Vicario et al. lfT3l PT4l l26l . 
each using a different models. The work of E |9l is concerned with verifying temporal properties for 
some classes of GSMPs, where the hard part is the treatment of the unbounded until operator which is 
achieved by putting restrictions on the number of concurrently active clocks. They also deal with com- 
putational issues related to symbolic computation of integrals over exponential-polynomial distributions. 
The work of ||T3j[l4j|26l is concerned with certain classes of stochastic Petri nets for which they develop 
a computational framework similar to ours which includes both exact and approximate computation of 
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the distributions. The major difference is that our formulation that separates the x and y variables, pro- 
vides for more sophisticated scheduling policies, such as those described in [1], that take clock values 
into consideration. 

The most urgent topics in our agenda are the implementation and the extension to cyclic DPA. From 
a computational standpoint, since we start with uniform distribution, all our density transformers re- 
sult in piecewise-polynomial functions that can be computed analytically using a mixture of zone-based 
algorithms and computer algebra tools. Of course, the obtained expressions will become increasingly 
complex due to case splitting and may require approximation. An alternative (but not scalable) way 
would be to work using discrete-time approximations of the duration distributions. The present results 
allow us to compute reachable symbolic states forward to any desired horizon, but since densities are 
much richer than zones, there is no immediate proof of convergence to a fixed point. Given that the 
density transformer can be phrased as a linear operator over state-related densities, we intend to inves- 
tigate functional analysis techniques like those used in ||6l to establish convergence and approximate 
termination. 

Acknowledgment: This work benefitted from discussions with E. Asarin and from numerous anony- 
mous referees. 
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